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Abstract 

A search for the lepton-flavor-violating decay of the tau into one charged lepton and two charged 
hadrons has been performed using 221.4 fb _1 of data collected at an e + e~ center-of-mass energy 
around 10.58 GeV with the BABAR detector at the PEP-II storage ring. In all 14 decay modes 
considered, the numbers of events found in data are compatible with the background expectations. 
Upper limits on the branching fractions are set in the range (0.7 - 4.8) x 10~ 7 at 90% confidence 
level. All results are preliminary. 
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1 INTRODUCTION 



Lepton-flavor violation (LFV) involving charged leptons has never been observed, and stringent 
experimental limits exist from muon branching fractions: B(fi — > e"f) < 1.2 x 10~ n [1] and B(fi — > 
eee) < 1.0 x 1CP 12 [2] at 90% confidence level (CL). Recent results from neutrino oscillation 
experiments [3] show that LFV does indeed occur, although the branching fractions expected in 
charged lepton decays due to neutrino mixing alone are probably no more than 10~ 14 [4]. 

In tau decays, the most stringent limit on LFV is B(t~ — > fi + e~e~) < 1.1 x 10~ 7 at 90% 
CL [5]. Many extensions to the Standard Model (SM), particularly models seeking to describe 
neutrino mixing, predict enhanced LFV in tau decays over muon decays with branching fractions 
from 10~ 10 up to the current experimental limits [6]. Observation of LFV in tau decays would 
be a clear signature of non-SM physics, while improved limits will provide further constraints on 
theoretical models. 

This paper presents preliminary results of a search for the decays 1 r~ — > f f h ± h~ where t 
represents an electron or muon and h represents a pion or kaon. In total there are 14 distinct final 
states consistent with charge conservation. The best existing limits on the branching fractions for 
these decay modes currently come from CLEO, and range from (2 — 8) x 10~ 6 at 90% CL [7]. 

2 THE BABAR DETECTOR AND DATASET 

The data used in this analysis were collected with the BABAR detector at the PEP-II asymmetric- 
energy e + e~ storage ring. The data sample consists of 205.3 fb" 1 recorded at y/s = 10.58 GeV and 
16.1 ftT 1 recorded at y/s = 10.54 GeV. With an estimated luminosity-weighted cross section for 
tau pairs of a TT = (0.89 ± 0.02) nb [8], this data sample contains over 390 million tau decays. 

The BABAR detector is described in detail elsewhere [9]. Charged-particle (track) momenta 
are measured with a 5-layer double-sided silicon vertex tracker and a 40-layer drift chamber inside 
a 1.5-T superconducting solenoidal magnet. An electromagnetic calorimeter (EMC) consisting of 
6580 CsI(Tl) crystals is used to identify electrons and photons, a ring-imaging Cherenkov detector 
and energy loss in the drift chambers are used to identify charged hadrons, and the instrumented 
magnetic flux return (IFR) is used to identify muons. 

A Monte Carlo (MC) simulation of LFV tau decays is used to study the performance of this 
analysis. Simulated tau-pair events including higher-order radiative corrections are generated using 
KK2f [8] with one tau decaying to one lepton and two hadrons with a 3-body phase space distribution, 
while the other tau decays according to measured rates [10] simulated with Tauola[11]. Final state 
radiative effects are simulated for all decays using Photos[12]. The detector response is simulated 
with GEANT4 [13], and the simulated events are then reconstructed in the same manner as data. 

3 ANALYSIS METHOD 

The analysis procedure is similar to that used in our published r~ — > £~£ + £~ analysis [5]. All 
possible lepton and hadron combinations consistent with charge conservation are considered, leading 
to fourteen distinct decay modes (e~ K + K~ , e~K + ir~, e~ir+K~, e~7r + 7r~, fi~K + K~ , fi~K + ir~, 
fi~7r + K~ , [i~ir + ir~ , e + K~K~ , e + K~ir~ , e + ir~7r~ , fi + K~K~ , fi + K~ir~ , ^ + 7r _ 7r _ ). The signature 
of this process is three charged particles, one identified as either an electron or muon, and each of 

throughout this paper, charge conjugate decay modes also are implied. 
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the other two identified as either a pion or kaon, with an invariant mass and energy equal to that 
of the parent tau lepton. 

3.1 SELECTION 

Candidate signal events in this analysis are required to have a "1-3 topology," where one tau decay 
yields three charged particles (3-prong), while the second tau decay yields one charged particle 
(1-prong). Four well reconstructed tracks are required with zero net charge, pointing towards 
a common region consistent with t + t~ production and decay The event is divided into two 
hemispheres using the plane perpendicular to the thrust axis, calculated from the observed tracks 
and unassociated EMC energy deposits in the center-of-mass (CM) frame. One hemisphere must 
contain exactly one track while the other must contain exactly three, defining the 1-3 topology. 
Pairs of oppositely charged tracks identified as photon conversions in the detector material with an 
e + e~ invariant mass below 30MeV/c 2 are ignored. 

One of the charged particles found in the 3-prong hemisphere must be identified as either 
an electron or muon candidate. Electrons are identified using the ratio of EMC energy to track 
momentum (E/p), the ionization loss in the tracking system (dE/dx), and the shape of the shower 
in the EMC. Muons are identified by hits in the IFR and small energy deposits in the EMC. Muons 
with momentum less than 0.5 GeV/c cannot be identified because they do not penetrate far enough 
into the IFR. Each of the other two charged particles found in the 3-prong hemisphere must be 
identified as either a pion or a kaon. 

The particle identification requirements alone are not sufficient to suppress certain backgrounds, 
particularly those from light quark qq production and SM r + r~ events, therefore additional selec- 
tion criteria are required. The selection requirements, most of which are the same for all 14 decay 
modes, are as follows: 

• no neutral clusters (photon candidates) with energy {E^ > 100 MeV) in the EMC. This 
restriction removes mostly qq backgrounds and some SM tau-pair events; 

• the polar angle of the missing momentum in the lab frame, O m iss> is required to be in the 
range 0.25 rad < O m i ss < 2.4 rad. This cut is effective at reducing two-photon and Bhabha 
backgrounds; 

• the total transverse momentum in the event in the CM frame, p^ M , must be greater than 0.2 
GeV/ c. The cut on p$f M is also effective against Bhabha and two-photon events; 

• the mass of the one-prong hemisphere, mi pr , calculated from the four- momentum of the 
track in the 1-prong hemisphere and the missing momentum in the event, is required to satisfy 
0.6 GeV/c 2 < m lpr < 1.9 GeV/c 2 for e^h ± h~ final states and 0.8 GeV/c 2 < m Vpr < 1.9 GeV/c 2 
for n^h^h^ final states. The one-prong mass requirement is particularly effective at removing 
qq backgrounds as well as the remaining two-photon contribution; 

• to remove any remaining Bhabha background, the momentum of the one-prong track in the 
CM frame, p™, is required to be less than 4.5 GeV/c for the e~7r + 7r~ and e + 7r _ 7r _ final 
states. 

In addition, particle ID vetoes are applied to specific selection channels. For the ehh decay 
modes, except for eKK, the 1-prong track must not be identified as an electron. This requirement 
is useful to reduce possible four-fermion eehh background events produced from two-photon or 
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Bhabha-like processes. For the decay modes with only pions (e _ 7r + 7r+ e + 7r~7r+ [i~tt + it~ , /i + 7r~7r~) 
or the mixed kaon-pion modes (e~K + ir~, e~ir + K~, e + K~ir~, [i~K + tt~, fi~ir + K~, fi + K~ir~), the 
lepton candidate also must not pass the kaon criteria. 

3.2 (AM, AE) OBSERVABLES 

To reduce backgrounds further, candidate signal events are required to have an invariant mass 
and total energy in the 3-prong hemisphere consistent with a parent tau lepton. These quantities 
are calculated from the observed track momenta assuming the corresponding lepton and hadron 
masses in each decay mode. The energy difference is defined as AE = - E™ m , where 
is the total energy of the tracks observed in the 3-prong hemisphere and -E^am ^ s ^ ne beam energy, 
both in the CM frame. The mass difference is defined as AM = M rcc — m T , where M rec is the 
reconstructed invariant mass of the three tracks and m T = 1.777 GeV/c 2 is the tau mass [14]. 

The signal distributions in the (AM, AE 1 ) plane are broadened by detector resolution and 
radiative effects. The radiation of photons from the incoming e + e~ particles before annihilation 
affects all decay modes, leading to a tail at low values of AE. Radiation from the final-state 
lepton produces a tail towards low values of AM which is more likely for electrons than muons. 
Rectangular signal regions are defined separately for each decay mode as follows. For all fourteen 
decay modes, the energy difference AE must be in the range [—100, +50] MeV. For the modes with 
muons, the mass difference AM is required to be in the range [-20, +20] MeV/ c 2 , while for the 
electron modes the range is [—30, +20] MeV/c 2 . 

These signal region boundaries are chosen to provide the smallest expected upper limits on the 
branching fractions in the background-only hypothesis. These expected upper limits for the signal 
box tuning are estimated using only Monte Carlo simulations, not candidate signal events. Figure 1 
shows the observed data for all fourteen selection channels in the (AM, AE) plane, along with the 
signal region boundaries and the expected signal distributions. To avoid bias, a blind analysis 
procedure was adopted with the number of data events in the signal region remaining unknown 
until the selection criteria were finalized and all systematic studies had been performed. 

3.3 SELECTION EFFICIENCY 

The efficiency of the selection for signal events is estimated with a MC simulation of LFV tau 
decays. About 40% of the MC signal events pass the initial 1-3 topology requirement. From 20% 
to 70% of these preselected events pass the particle ID criteria, depending upon the final state, and 
70% fall within the signal region in the (AM, AE) plane. The final selection requirements accept 
from 17% to 27% of these remaining signal MC events. The final efficiency for signal events to be 
found in the signal region is shown in Table 1 for each decay mode and ranges from 2.1% to 3.8%. 
This efficiency includes the 85% branching fraction for 1-prong tau decays. 

The particle ID efficiencies and misidentification probabilities are not taken from Monte Carlo, 
but rather measured using tracks in kinematically-selected data samples. These values are param- 
eterized as a function of particle momentum, polar angle, and azimuthal angle in the laboratory 
frame. The control samples used to characterize the lepton ID performance include radiative 
Bhabha, radiative [i + fi~, two-photon e + e~£ + £™, and J/ip — > events. The control sample 

used to characterize the hadron ID performance is the decay D* — > D°tt, D — > Kit. These data- 
derived efficiencies are then used to determine the probability that a simulated MC particle will be 
identified (or misidentified) as an electron, muon, pion, or kaon. 
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Figure 1: Observed data shown as dots in the (AM, AB) plane and the boundaries of the signal 
region for each decay mode. The dark and light shading indicates contours containing 50% and 
90% of the selected MC signal events, respectively. The regions shown in Fig. 2 are indicated by 
dashed lines. The vertical dotted lines show the lower AM boundary of the grand sideband (GS) 
region for the fj,hh final states. The other GS boundaries are the plot borders. 
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Table 1: Efficiency estimates, the number of expected background events (iV^) in the signal region, 
the number of observed events (N f, s ) in the signal region, and the 90% CL upper limit for each 
decay mode. 



Mode 


Eff. [%] 


x * oga 


N nhe 

x T DOS 


UL at 90% CL 


e~K + K- 


3.77 ± 0.16 


0.22 ± 0.06 





1 A 
1.4 


1U 




3.08 ± 0.13 


0.32 ± 0.09 


o 


1. I 


10 




3 10 ± 13 


14 ± 06 


1 


6.1 


1U 


€~ 7T~^ 7T~ 


3 30 ± 15 


81 ± 15 


o 


l.z 


1U 




2.16 ± 0.12 


24 ± 08 


o 


2.5 


io- 7 




2.97 ±0.16 


1.67 ±0.32 


2 


3.2 


io- 7 




2.87 ±0.16 


1.04 ± 0.20 


1 


2.6 


io- 7 




3.40 ±0.19 


2.99 ± 0.42 


3 


2.9 


io- 7 


e+K-R- 


3.85 ±0.16 


0.04 ± 0.04 





1.5 


io- 7 




3.19 ±0.14 


0.16 ±0.06 





1.8 


io- 7 




3.40 ±0.15 


0.41 ±0.10 


1 


2.7 


io- 7 




2.06 ±0.11 


0.07 ±0.10 


1 


4.8 


io- 7 


/i, + K~ir~ 


2.85 ±0.16 


1.54 ±0.28 


1 


2.2 


io- 7 




3.30 ±0.18 


1.46 ± 0.23 





0.7 


io- 7 



The lepton identification has been designed to give very low mis-identification rates at the 
expense of some efficiency. The electron ID is expected to be 81% efficient in sig nal fFfci/r events 
with a mis-ID rate of 0.1% for pions and 0.2% for kaons expected in generic t + t~ events. The 
muon ID is 44% efficient for signal events, with a mis-ID rate of 1.0% for pions and 0.4% for kaons. 
The hadronic identification is designed to classify the hadronic candidates as pions or kaons, but is 
not intended to distinguish hadrons from leptons. Since the dominant backgrounds contain many 
pions (and some kaons), stricter requirements on identifying hadrons do not improve the analysis. 
The pion ID is 92% efficient with a mis-ID rate of 12% for kaons, while the kaon ID is 81% efficient 
with a 1.4% mis-ID rate for pions. The pion ID criteria also has a very high mis-ID rate for muons 
(98%) and electrons (38%) which means that the n~ir + ir~ channel is also rather sensitive to other 
LFV tau decay modes like r — > [i~fi + fi~. 

3.4 BACKGROUND ESTIMATES 

There are two main classes of background remaining after the selection criteria are applied: low 
multiplicity qq events (mainly continuum light-quark production) and SM r + r~ events. These 
background classes have distinctive distributions in the (AM, AE) plane: qq events tend to populate 
the plane uniformly, while t + t~ backgrounds are restricted to negative values of both AE and AM. 
Backgrounds from Bhabha and [x + \x~ events (which were important for our r — > ill analysis [5]) 
are found to be negligible in the t~ — ► £^h^h~ decay modes. The two-photon backgrounds are 
also found to be negligible, although studies show they would look very similar to qq backgrounds. 

For each background class, a probability density function (PDF) describing the shape of the 
background distribution in the (AM, AE) plane is determined by fitting an analytic function to the 
Monte Carlo prediction as described in more detail below. These two PDFs are then combined with 
coefficients determined by a fit to the observed data in the (AM, AE) plane in a grand sideband 
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(GS) region which excludes the signal region. The resulting function describes the event rate seen 
in the GS region and is then used to predict the expected background rate in the signal region. 

The GS region, shown in Fig. 1, is defined as the rectangle bounded by (— 700MeV/c 2 < AM < 
400MeV/c 2 ) for e^h ± h~ final states, (-400MeV/c 2 < AM < 400MeV/c 2 ) for /j^h ± h~ final states, 
and (— 700 MeV < AE < 400 MeV) for both, excluding the signal region. For the qq backgrounds, 
an analytic PDF is constructed from the product of two PDFs Pm> and Pe>, where Pm'(AM') is 
a Gaussian and P E '{AE') = (1 - x/y/1 + x 2 )(l + ax + bx 2 + cx 3 ) with x = (AE' - d)/e[15]. The 
(AM', AE') axes have been rotated slightly from (AM, AE) to better fit the expected distributions. 
The resulting qq PDF is described by eight fit parameters, including the rotation angle, which are 
determined by fits to MC qq background samples for each decay mode. The r + r _ PDF function 
Pm'(AM') is the sum of a Gaussian and a bifurcated Gaussian, while the functional form of 
Pe'(AE') is the same as the qq PDF above. To properly model the kinematic limit in tau decays, a 
similar coordinate transformation is performed to get AM' and AE' , except that the axes are not 
required to remain orthogonal. 2 This method reproduces well the slightly wedge-shaped background 
distribution seen from tau decays in Figure 1 for the [mir final states. In total there are 12 free 
parameters describing this PDF, and all are determined by fits to MC t + t~ samples, including the 
two coordinate transformation angles. 

With the shapes of the two background PDFs determined, an unbinned maximum likelihood fit 
to the data in the GS region is used to find the expected rate of each background type in the signal 
region, as shown in Table 1. The PDF shape determinations and background fits are performed 
separately for each of the fourteen decay modes. Figure 2 shows the data and the background 
PDFs for values of AE in the signal range. 

4 SYSTEMATIC STUDIES 

The largest systematic uncertainty for the signal efficiency is due to the uncertainty in measuring the 
particle ID efficiencies. This uncertainty 3 is determined from the statistical precision of the particle 
ID control samples, and ranges from 0.7% for e~7r + 7r~ to 3.8% for jjT K + K~ . The modeling of 
the tracking efficiency contributes an additional 2.5% uncertainty, while the statistical uncertainty 
of the MC signal samples ranges from 1% to 2%. All other sources of uncertainty are found to be 
small, including the modeling of radiative effects, track momentum resolution, trigger performance, 
observables used in the selection criteria, and knowledge of the tau 1-prong branching fractions. 
The efficiency has been estimated using a 3-body phase space model and no uncertainty is assigned 
for possible model dependence. The selection efficiency is found to be uniform within 20% across 
the Dalitz plane, provided the invariant mass for any pair of particles from the LFV decay is less 
than 1.4GeV/c 2 . 

Since the background levels are extracted directly from the data, systematic uncertainties on 
the background estimation are directly related to the background parameterization and the fit 
technique used. The finite data available in the GS region used to determine the background rates 
is the largest uncertainty and varies from 14% to 140% depending upon the decay mode. Additional 
uncertainties of 10% are estimated by varying the fit procedure and changing the functional form 
of the background PDFs. The uncertainty on the branching fraction of SM tau decays with one 
or two kaons in the final state contributes 0-14% to the uncertainty on the estimated background. 

2 The transformation is AM' = cos AM + sin j3\ AE and AE' = cos f3 2 AE - sin ftAM. The coefficients (3i and 
02 are also fit from the predicted MC background distributions. 
3 A11 uncertainties quoted in the text are relative. 
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Figure 2: Distribution of AM for data (solid histogram) and background PDFs (solid curves) for 
events with AE in the range [—100, +50] MeV as indicated in Fig. 1. Expected signal distributions 
are shown (dashed histogram) for a branching fraction of 5 • 10 -7 . 
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Cross checks of the background estimation were performed by considering the number of events 
expected and observed in sideband regions immediately neighboring the signal region for each decay 
mode as shown in Table 2. Good agreement is seen between the observed data and background 
expectations. 

Table 2: The number of events expected (left) and observed (right) in the regions just neighboring 
the signal region. These "neighbor boxes" have the same dimension as the original signal region for 
each decay mode, but have been offset by the width or height of the signal region in the (AM, AE) 
plane. 



Mode 


left 


top 


right 


bottom 


e-K + K- 


0.22 





0.05 


1 


0.24 


1 


0.30 





e-K+Tr- 


0.31 





0.10 





0.31 





0.35 





e~ir + K~ 


0.15 





0.05 





0.13 





0.18 





e~ir + ir~ 


0.89 





0.36 


1 


0.76 





0.93 







0.30 


1 


0.04 





0.21 





0.33 







1.81 





0.16 





1.37 


2 


1.79 


1 


li-ir + K- 


1.30 


1 


0.14 





0.84 





1.09 


3 




4.66 


10 


0.97 





2.09 


1 


2.68 





e + K-R- 


0.05 





0.00 





0.03 





0.05 







0.17 





0.07 





0.14 





0.19 







0.43 





0.13 





0.37 





0.43 


1 




0.08 





0.01 





0.06 





0.08 







1.87 


2 


0.23 





1.26 





1.83 







2.42 


3 


0.52 





1.15 


1 


1.54 






5 RESULTS 

The numbers of events observed (N b s ) and the background expectations (N^gd) are shown in 
Table 1, with no significant excess found in any decay mode. Upper limits on the branching fractions 
are calculated according to Bfj L = Njj L /(2e£a TT ), where Nfj° L is the 90% CL upper limit for the 
number of signal events when N b s events are observed with N^gd background events expected. The 
values e, C, and a TT are the selection efficiency, luminosity, and r + r~ cross section, respectively. 
The estimates of C = 221.4 fb _1 and a TT = 0.89 nb are correlated, 4 and the uncertainty on the 
product Ca TT is 2.3%. The upper limits on the branching fraction have been calculated including 
all uncertainties using the technique of Cousins and Highland [16] following the implementation of 
Barlow [17]. The 90% CL upper limits on the r~ — > £^ : h ± h~ branching fractions, shown in Table 1, 
are in the range (0.7 — 4.8) x 10 -7 . These limits represent an order of magnitude improvement over 
the previous experimental bounds [7]. 

4 The luminosity is measured using the observed fi + n~ rate, and the and t + t~ cross sections are both 

estimated with KK2f . 
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